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Abstract 

We are interested in modeling some two-level population dynamics, resulting from 
the interplay of ecological interactions and phenotypic variation of individuals (or 
hosts) and the evolution of cells (or parasites) of two types living in these individuals. 
The ecological parameters of the individual dynamics depend on the number of cells 
of each type contained by the individual and the cell dynamics depends on the trait 
of the invaded individual. 

Our models are rooted in the microscopic description of a random (discrete) pop- 
ulation of individuals characterized by one or several adaptive traits and cells charac- 
terized by their type. The population is modeled as a stochastic point process whose 
generator captures the probabilistic dynamics over continuous time of birth, mutation 
and death for individuals and birth and death for cells. The interaction between indi- 
viduals (resp. between cells) is described by a competition between individual traits 
(resp. between cell types). We look for tractable large population approximations. 
By combining various scalings on population size, birth and death rates and mutation 
step, the single microscopic model is shown to lead to contrasting nonlinear macro- 
scopic limits of different nature: deterministic approximations, in the form of ordinary, 
integro- or partial differential equations, or probabilistic ones, like stochastic partial 
differential equations or superprocesses. The study of the long time behavior of these 
processes seems very hard and we only develop some simple cases enlightening the 
difficulties involved. 



Key-words: two-level interacting processes, birth-death-mutation-competition point 
process, host-parasite stochastic particle system, nonlinear integro-differential equations, 
nonlinear partial differential equations, superprocesses. 

1 Centre de Mathematiques Appliquees, UMR C.N.R.S. 7641, Ecole Polytechnique, 91128 Palaiseau 
Cedex, France. Email: sylvie.meleard@polytechnique.edu 

2 Institut fur Mathematik der Universitat Potsdam, Am Neuen Palais 10, 14469 Potsdam, Germany. 
Email: roelly@math.uni-potsdam.de 

3 0n leave of absence Centre de Mathematiques Appliquees, UMR C.N.R.S. 7641, Ecole Polytechnique, 
91128 Palaiseau Cedex, France. 



1 



1 Introduction 



In this paper, we are interested in describing the adaptive effects in a host-parasite system. 
We model two-level population dynamics resulting from the interplay of ecological inter- 
actions and phenotypic variation of individuals (or hosts) and the evolution of cells (or 
parasites) of two types living in these individuals. In one hand, the ecological parameters 
of the individual dynamics depend on its number of cells of each type. In another hand, the 
cells develop their own birth and death dynamics and their ecological parameters depend 
on the trait of the invaded individual. 

We consider more precisely the following two-level population. The first level is composed 
of individuals governed by a mutation-birth and death process. Moreover each individual 
is a collection of cells of two types (types 1 and 2) which have their own dynamics and 
compose the second level. The model can easily be generalized to cells with a finite 
number of different types. We denote by n\ (resp. by n|) the number of cells of type 
1 (resp. of type 2) living in the individual i. This individual is moreover characterized by 
a continuous quantitative phenotypic trait x l . The individual i can be removed or copied 
according a birth-and-death process depending on x 1 ,n\,n l 2 . An offspring usually inherits 
the trait values of her progenitor except when a mutation occurs during the reproduction 
mechanism. In this case the offspring makes an instantaneous mutation step at birth to new 
trait value. The death of an individual can be natural or can be due to the competition 
exerted by the other individuals, for example sharing food. This competition between 
individuals through their traits will induce a nonlinear convolution term. The cells in 
the individual i also reproduce and remove according to another birth-and-death process, 
depending on x 1 ,n\,n^. At this second level, cell competition occurs and its pressure 
depends on the invaded individual trait. 

Our model generalizes some approach firstly developed in Dawson and Hochberg |3J and 
in Wu |12j . In these papers, a two- level system is studied: individuals and cells follow 
a branching dynamics but there is no interaction between individuals and between cells. 
Thus all the specific techniques these authors use - as Laplace transforms - are no more 
available for our model. 

In this paper, we firstly rigorously construct the underlying mathematical model and prove 
its existence. Thus we obtain moment and martingale properties which are the key point 
to deduce approximations for large individual and cell populations. By combining various 
scalings on population size, birth and death rates and mutation step, the single microscopic 
model is shown to lead to contrasting macroscopic limits of different nature: deterministic 
approximations, in the form of ordinary, integro- or partial differential equations, or prob- 
abilistic ones, like stochastic partial differential equations or superprocesses. The study 
of the long time behavior of these processes seems very hard and we only develop some 
simple cases enlightening the difficulties involved. 
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2 Population point process 



2.1 The model 

We model the evolving population by a stochastic interacting individual system, where 
each individual i is characterized by a vector phenotypic trait value x % and by the number 
of its cells of type 1, n\, and of type 2, n 2 . The trait space X is assumed to be a compact 
subset of R d , for some d > 1. We denote by M F = M F (X x N x N) the set of finite 
non-negative measures on X x N x N, endowed with the weak topology. Let also M be 
the subset of M F consisting of all finite point measures: 

M = {E^W 2 )> x * g K> n 2) G N x N, 1 < i < I, I G n| . 

Here and below 5r XjnijJl2 \ denotes the Dirac mass at (x, n\,ri2). In case where 1 = 0, the 
measure is the null measure. 

Therefore, for a population modelled by v = Yli=i ^(x i n\ n;,)' the t°t a l number of its indi- 
viduals is (v, 1) = I and, if we denote by n := n\+ n 2 the number of cells of an individual 
(irrespective of type), then (u, n) = J2i=i( n \ + n \) ls *he total number of cells in the 
population v. 

Let us now describe the two-level dynamics. Any individual of the population with 
trait x and cell state (ni,n2) follows a mutation-selection-birth-and-death dynamics with 

• birth (or reproduction) rate B(x,n\,n2), 

• the reproduction is clonal with probability 1 — p(x) (the offspring inherits the trait 

x), 

• a mutation occurs with probability p(x), 

• the mutant trait x + z is distributed according the mutation kernel M(x, z) dz which 
only weights z such that x + z G X, 

• death rate D(x, n\, 112) + a(x, n±, 112) Ylj=i U{x — x^). 

Thus the interaction between individuals is modeled by a comparison between their re- 
spective trait values described by the competition kernel U. By simplicity, the mutations 
parameters p and M are assumed to be only influenced by the trait x. They could also 
depend on the cell composition (m, n.2) without inducing any additional technical difficulty. 

Any cell of type 1 (resp. of type 2) inside an individual with trait x and cell state (ni,n2) 
follows a birth-and-death dynamics with 

• birth rate bi(x), (resp. 62(2)), 

• death rate d\(x) + (3i(x)(niXu + n 2 Ai 2 ), (resp. d 2 (x) + /3 2 (x)(niA 2 i + 1^2X22))- 
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The nonnegative parameters An, A22, A12, A21 quantify the cell interactions. The rate 
functions b±, 62, di, d 2 , /3 2 are assumed to be continuous (and thus bounded on the 
compact set X). 

The population dynamics can be described by its possible transitions from a state v to the 
following other states: 

1 - Individual dynamics due to an individual with trait x and cell state (711,7x2): 
v ^ f + £(x,m,ra 2 ) with rate B(x,n 1 ,n 2 )(l - p(x)) ; 

v 1 y v — S( x ni Jl2 j with rate D(x, n\,n 2 ) + a(x, n±, n 2 ) U(x — x J ) ; 

1/ i-> 1/ + (5( x+2iniiri2 ) with rate B(x, ni,ri2) p(x), where z is distributed following M(x, z) dz. 

2 - Cell dynamics: 

v ^ v + *( x>ni+ i )n2 ) - i( Iiniinj ) with rate 61 (s) ; 

f ^ zv + <5( Xjni)fl2+ i) - 5(x,ni,n 2 ) with rate b 2 (x) ; 

" ^ " + ^(x,m-i,ri2) ~ S (x, ni ,n 2 ) with rate d x {x) + /3i(x)(Anni + Ai 2 n 2 ) ; 

v ^ i y + <5(x,ni,n 2 -i) - S (x,m,n 2 ) with rate d 2 (x) + /3 2 (x)(X 2 in 1 + A 22 n 2 ). 

Let us now prove the existence of a cadlag Markov process (ft)t>o belonging to D(R + , A4) 
modeling the dynamics of such a discrete population. More precisely, we consider 

l(t) 

^ = Y1 6 (X'(t),Nl(t),N^(t)) (2- 1 ) 
i=l 

where I(t) G N stands for the number of individuals alive at time t, X 1 (t), AT^*** (i) £ X 
describes the traits of these individuals at time t and AT^(t), iV-j^ (i) (resp. N^t), ATg^ (t)) 
are the numbers of cells of type 1 (resp. of type 2) for the individuals alive at time t. 
To write down the infinitesimal generator of u, we need an appropriate class of test func- 
tions. For bounded measurable functions 0, /, g±, g 2 defined respectively on R, M. d , N and 
N, 4>f gi92 is given by 

fygigii 1 ') := <P(< vJ9i92 >) = <P [ f(x)g 1 (n 1 )g 2 (n 2 )u(dx,dn 1 ,dn 2 ) 




The infinitesimal generator L of the Markov process (u t ,t > 0) applied to such function 
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I 

i=l 

+ E / tt^fam) + ftf + z) gi (n\)g 2 (n 2 )) - ^(u,fg 1 g 2 }))B(x\n\,nl)p(x l )M(x\z)dz 
i=l ^ 

+ ^f 2 ) - /(^)5iK)52K 2 )) - /5i52)))(-D(^,ni,4) + a(x i ,nl,n 4 2 )f/ * v(x\n[,n 2 )) 

i=l 

+ J>«f,/Si02> + /(^)(5iK + 1) - 3lK))ff 2 (n|)) - /<&<fc»)&l 
i=l 

+ /<?i<72> + /(^)5lK)(52(4 + 1) - 52(4))) - <K(v, f gi g 2 )))b 2 (x l )n 2 

i=l 
/ 

+ J>«f,/SiS2> + /(x J )(5iK - 1) - 3lK))ff 2 (ni)) - <K(v,fgm))) 
i=l 

(^(x*) + /M^XAnni + A 12 n 2 )K 

/ 

+ J>«i>,/SiS2> + /(x i )gi(n t 1 )( 52 (n l 2 - 1) - 52 (n 2 ))) - <f>{(v,fgm))) 
i=i 

(d 2 (x l ) + P 2 (x i )(\nn\ + A 22 n 2 ))n 2 . (2.3) 

The three first terms of (j2.3H capture the effects of births and deaths of individuals of the 
population and the for last terms that of the cells. The competition makes the death terms 
nonlinear. 

2.2 Process construction 

Let us give a pathwise construction of a Markov process admitting L as infinitesimal 
generator. 

Assumptions (HI): 

There exist constants B, D, G a, U and C and a probability density function M on M. d 
such that for x, z 6 X, n\,n 2 G R+, 

B(x,ni,n 2 ) < B ; 
D(x, m, n 2 ) < D (m + n 2 ) = D n ; 
a(x, ni,n 2 ) < a (n\ + n 2 ) = an ; 
U{x) < U, M(x,z) < CM{z). 

Remark that the jump rate of an individual with n cells in the population v is then 
upper-bounded by a constant times n (1 + (v, 1)) and that the cell jump rate of such 
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individual is upper-bounded by a constant times n(l + n). Thus the model presents a 
double nonlinearity since the population jump rates may depend on the product of the size 
of the population times the number of cells and quadratically on the number of cells. 

Let us now give a pathwise description of the population process (ut)t>Q- 

Notation 2.1 We associate to any population state v = ^[ =1 5^ x i n i n i j £ A4 the triplet 
H l (y) = (X 1 (u) , Nl(u) , N^v)) as the trait and state of the ith-individual, obtained by 
ordering all triplets with respect to some arbitrary order on M d x N x N ( for example the 
lexicographic order). 

We now introduce the probabilistic objects we will need. 

Definition 2.2 Let (£1, J 7 , P) be a probability space on which we consider the following 
independent random elements: 

(i) a M. -valued random variable vq (the initial distribution), 

(ii) A Poisson point measure Q(ds, di, dz, d9) on R + xN*x^fx IR + with intensity measure 

ds (j2k>i M(z)dzd6 . 

Let us denote by {Tt)t>0 the canonical filtration generated by vq and Q. 

Let us finally define the quantities 6\(s), 9 2 (s), 9\(s), 0\(s), 9\(s), 9\(s), 9\(s) related to 
the different jump rates at time s as: 





9\{s) 


= B{H\v s . 


.))(l-p(X>.-))); 




9\(s) 


= B{H\u s - 




e\{s) - 


0\{s) 


= D{IP{y 8 . 


_))+<*(#>._)) U*v s _(X\v s _)); 


0\(s) - 


0\(s) 


= &l(X> a - 


-)) m V3 -y, 




0\{s) 


= H^iys- 


-)) m»s-y, 


- 




= di(*> a . 


_)) +/3 1 (X> s „))(iV7(^_)A 11 +Ni(v 8 -)\ 12 ) Ni(y s _ 


9\{s) - 


oi(s) 


= d 2 (X i (v s _ 


.)) + P 2 {X\u s _))(N{{u s _)\ 21 + N l 2 (v s _)\ 22 ) N l 2 \u s _ 



(2.4) 

We finally define the population process in terms of these stochastic objects. 

Definition 2.3 Assume (HI). A (F t )t>Q-0'dapted stochastic process v = (vt)t>o * s called 
a population process if a.s., for all t > 0, 



6 



n - + / \ S^ X i(u s -),Ni(u s -),Ni{u s ^))^i<(u 3 -^)} 1 {e<eu s )} 

J(0,t]xN*xXxR+ I 1 2 

+ ^(x i (^_)+^,Af 1 i K_),^(i/ s _)) 1 {i<K-,i)} 1 {^( s )<e<^( s )} 

~ ^( l / s _),ivi(^_),Ar|(v s _)) 1 {i<<^-,i>} 1 {0i( S )<0<ei( S )} 



H 5 



(^(X*(u s ^),N{(v a -),NZ(v s -)+l) ~ 5(Xi(v a ^),N{(v s ^),N& s -))^ 1 {i<{vs-A)} 1 {6i(s)<6<6i(s)} 

(^(x^u s ^),Ni(u s .)-i,N^u a .)) ~ 5 (x i K_),AriK_),7V|K_))^) 1 {j<(^-,i)} 1 {e|( s )<e<e> ( s )} 



Q(ds, di, dz, dO) 
(2.5) 

Let us now show that if v solves (|2.5p . then v follows the Markovian dynamics we are 
interested in. 

Proposition 2.4 Assume (HI) and consider a process (vt)t>o defined by K2. 5j) such that 
for all T > 0, E(sup i<T (ft, l) 3 ) < +oo and E(sup t<T (fj, n 2 )) < +oo. T/ien (ff)t<o a 
Markov process. Its infinitesimal generator L applied to any bounded and measurable maps 
<fifgig2 ■ M i— > R and v £ M satisfies &2.3\) . In particular, the law of (vt)t>o does not 
depend on the chosen order in Notation \2.1\ 

Proof The fact that (ft)t>o is a Markov process is immediate. Let us now consider a 
function 4>f gig2 as in the statement. Using the decomposition f)2.5[) of the measure z/j and 
the fact that 

<t>fBiga(yt) = <l>f9l92( U o) +^2(<t>f 9l g 2 (v S - + Os - !>,-)) - 0/9192 (^-)) a - s -> ( 2 - 6 ) 

we get a decomposition of <t>fgi_g 2 (yt)- 

Thanks to the moment assumptions, 4>fgig2( l/ t) is integrable. Let us check it for the non- 
linear individual death term (which is the more delicate to deal with): 



E 



{HWs- - &(Xi{v s YNUv 3 YNi{v 3 \)i f9l9z) ~ <t>(Ws-, /Slfti)) l{i<<i/._,l)} 
(0,t]xN*xA'xR + 1 2 
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{ei(s)<e<ei(s)}Q( ds ' di > dz ' dd ) 



E / Ws, [<t>(Ws,fgi92) - f(x)gi(ni)g 2 (n 2 )) - cj){{y s , f gig 2 ))(D(x,n u n 2 ) + a(x,n 1 ,n 2 ) U*v s {x)))d. 



s 



'0 
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Since <f> is bounded and thanks to Assumption (HI), the right hand side term will be finite 
as soon as 

1 I sup((i/ t ,n) + (i/ t ,ra)(i/ t ,l)) < oo. 
\t<T J 

Remark firstly that (f,n) < (v,n 2 ). Moreover we get n(u, 1) < l/2(n 2 + (u, l) 2 ) and thus 

(u, n) (u, 1) < l/2((i/, n 2 + (u, l) 2 )) = l/2((u, n 2 ) + (u, if). 

The moment assumptions allow us to conclude and to show that the expectation is differ- 
entiable in time at t = 0. It leads to (12.31). □ 



Let us show existence and moment properties for the population process. 

Theorem 2.5 Assume (HI). 

(i) If E((i/o,l)) < +oo ; then the process (vt)t introduced in Definition \2.3\ is well 
defined on R + . 

(ii) Furthermore, if for some p > 1, E ((1/0, 1) P ) < +00, then for any T < 00, 

E( sup (v t ,l) P ) < +00. (2.7) 
te[o,T] 

(Hi) If moreover E (^i/o,n 2 )) < +00, then for any T < oo, 

E( sup (v t ,n 2 )) < +00. (2.8) 
te[0,T] 

Proof We compute </>(< i/£, 1 >) using (|2,5p and (|2.6p for / = 51 = 52 = 1 ; we get 

(/)(< v u 1 >) = 4>(< i/ , 1 >) + / \ (<f>(< i/ s _, 1 > +1) - 0(< z/ s _, 1 >)) lf e<e i (s) \ 

J(0,t]xN*xXxR+ I 



+ (<£(< ^s-' 1 > -1) -0(< ^s-' 1 >)) 1 {^(s)<0<0i(s)} j 1 {i<{^-,l)} ( 3( ,is ' (ii ' d2; ' d61 ) 

(2.9) 

and for gi(n\) = n\, 

< u t , m > =< 1/0, ni > + / i ( 1 {9<ei( s )> " 1 {ei( s )<e<eu s )i ) 

+ 1 {ei( s )<e<ei( s )} ~ 1 {^( s )<e<^( s )}} 1 {i<(^-,i>} ( 3( d ' s '^' d2; ' d6 ')- ( 2 - 10 ) 
A similar decomposition holds for < i>t, n 2 >■ 

The proof of (i) and (ii) is standard and can easily be adapted from [7]: we introduce 
for each integer k the stopping time = inf {t > 0, (i/f, 1) > A;} and show that the 
sequence(rfc)fc tends a.s. to infinity, using that 

sup (i/ a ,l) < (i/ ,l) + / l{i<(i/ s _,l>} l/0<0i( a uQ(ds,di,dz,d0), 

s£[0,tAT k ] J(0,tAT k ]xN*xXxR+ 



s 



and the estimates of moments up to time r/% deduced from the latter and Assumption (HI) 
and Gronwall's lemma. 

Further, one may build the solution (vt)t>o step by step. One only has to check that the 
sequence of jump instants (Tj.) goes a.s. to infinity as k tends to infinity, which follows 
from the previous result. 

The proof of (Hi) follows a similar argument with t\ := inf ji > 0, (i^nf) > From 



sup (u 8 ,nf) < {u ,nl) + / l{i<(^ s _,i» 

se[0,tAT^] J(0,tAri]xN*xA'xR+ 

(iVi(^_)) 2 l {( K0i (s)} + (2iVj( I / s _) + l)l {9 , (s) ^ (s)} |Q(d s ,di,^,^), 

and E ((Vo,n 2 )) < +00 and (ii) since 2ni + 1 < n\ + 2, we firstly get, using Assumption 
(HI) and Gronwall's lemma, that 

E( sup (u t ,nj))<C T . 

t6[0,TAT^] 

Then we deduce that tends to infinity a.s. and that E(sup tg [ 0j r] (ff,n 2 )) < +00. The 
same is true replacing m by n.2- □ 

2.3 Martingale Properties 

We finally give some martingale properties of the process (vt)t>o, which are the key point of 
our approach. For measurable functions /, (71,52 > let us denote by Ff g the function defined 
on Mp by 

Ff 9 (v) :=< u,fgig 2 > ■ 



Theorem 2.6 Assume (HI) together with E ((uo, l) 3 j < +00 and E ((Vo,n 2 )) < +00. 

(i) For all measurable functions 0, /, 51,(72 swc/i i/iai 

l^/ffifla W I + l^/fli^ Ml < C(l + !> 3 + (^>« 2 >); ^ e process 

tfgigafa) - <Pfg ig2 (vo) - I Lcp fgig2 (v s )ds (2.11) 

JO 

zs a cadlag (Tt)t>Q-Tnartingale starting from 0, where L(pf gig2 has been defined in A2.3\) . 

(ii) For all measurable bounded functions f, 51,(72; the process 

M[ 9 = (u u fg l92 ) - (z/ , / 5 i52> - / LF fg (u s )ds (2.12) 

JO 
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is a cadlag square integrable (J~t) t>o -martingale starting from 0, where 
LF fg (u) = \[ B(x, m, n 2 )(l - p(x, ni,n 2 )) - (D(x, ni,n 2 ) + a(x, ni,n 2 )U * v{x)) 

f{x)g 1 {n 1 )g 2 (n 2 ) 
+ p(x,ni,n 2 )B(x,n 1 ,n 2 ) f(x + z)gi(n{)g 2 (n 2 )M(x,ni,n 2 , z)dz 



+ f(x)(gi(ni + 1) - fli(ni)) g 2 (n 2 )bi(x)ni + f{x)g\{n\) (g 2 (n 2 + 1) - g 2 {n 2 ))b 2 (x)n 2 
+ f(x)(gi(ni - 1) - ffi(ni))f/ 2 (n 2 )(di(a;) + /3i(x)(X 11 n 1 + X 12 n 2 ))nt 



+ f{x)gi(n 1 )(g 2 (ni - 1) - g 2 {n 2 )) (d 2 (x) + f3 2 {x)(\ 2 \n\ + \ 22 n 2 ))n 2 jv(dx,dni,dn 2 ). 

(2.13) 

Its quadratic variation is given by 

(M f9 ) t = / / <M (1 - p(x,n 1 ,n 2 ))B(x,n 1 ,n 2 ) + (D(x,ni,n 2 ) + a(x, m, n 2 )/7 * v s (%)) 

JO JXxN 2 I V 

f 2 {x)gl(n x )gl(n 2 ) 
+ p(*. ni ,n 2)B (x, ni , „ 2 ) //»(. + ^M 9 I(» 2 )M(*, „ t , „ 2 , 

+ / 2 (^)(5'i(«i + 1) - 5i("-i)) 2 5' 2 ("-2)&i(2;)ni 
+ f 2 (x)gl(n 1 )(g 2 (n 2 + 1) - # 2 (n 2 )) 2 b 2 (x)n 2 

+ / 2 (^)(5'i("-i - 1) - 5i("-i)) 2 5' 2 ("'2)(^i(a;) + /3i(x)(Anni + Ai 2 n 2 ))m 
+ f 2 (x)gl(n 1 )(g 2 (n 1 - 1) - g 2 (n 2 )) 2 (d 2 (x) + f} 2 (x){\ 21 ni + A 22 n 2 ))n 2 J>zA,(dx, dm, dn 2 )ds. 



(2.14) 

Proof The martingale property is immediate by Proposition 12.41 and Theorem 12.51 Let 
us justify the form of the quadratic variation process. Using a localization argument as 
in Theorem 12. 5[ we may compare two different expressions of (ut, fg\g 2 ) 2 - The first one 
is obtained by applying (12. lip with (j){v) := (v, f gig 2 ) 2 ■ The second one is obtained by 
applying Ito's formula to compute (z^, fg\g 2 ) 2 from (|2,12p . Comparing these expressions 
leads to (I2.14p . We may let go the localization stopping time sequence to infinity since 
E((v , l) 3 ) < +oo and E ((^ ,n 2 )) < +oo. Indeed, in this case, E((M^ 9 ) t ) < +oo thanks 
to Theorem 12.51 and to the proof of Proposition 12.41 □ 



3 Deterministic large population approximations 

We are interested in studying large population approximations of our individual-based sys- 
tem. We rescale the size of individual population by K and the size of the cell populations 
by K\ respectively K 2 . With k, = (K, K\, K 2 ), the process of interest is now the Markov 
process (Y t K )t>o defined as 

1 In® 

Y t K = wY, 5 n w n w G M ^ x x M + x M +) 
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in which cells of type 1 (resp. of type 2) have been weighted by (resp. by -^j) and 
individuals by The dynamics of the process (Xj. (t), N[ K (t),N 2 K (t)) is the one described 
in Section 2 except some coefficients are depending on the scaling k as described below. 
The individual dynamics depends on B K , p K , M K , D K , a K , U K which are assumed to satisfy 
the assumptions (Hi) of the section 2 for any fixed k. 

Notation: We say that k — > oo when the three parameters K, Ki,K 2 tend to infinity. 
Assumptions (H2): 

1) There exist continuous functions B, D and a on X x M + x R + such that 

lim sup \B K (x,K 1 y 1 ,K 2 y2) - B(x,yi,y 2 )\ + \D K (x, K x yi, K 2 y 2 ) - D(x,yi,y 2 )\ = 0, 

K ^°° x,yi,y 2 

lim sup \a K (x,K 1 y 1 ,K 2 y 2 ) - a(x,y x ,y 2 )\ = 0. (3.15) 

We assume that the functions B, D and a satisfy Assumption (HI). 

2) The competition kernel U K satisfies 

U K (x) = ^M, (3.16) 

where U is a continuous function. 

3) The others parameters p K = p and M K = M stay unchanged, as also the cell ecological 
parameters: = b\, 6 2 ,k = b 2 , = di,d2,n = d 2 , f3 1<K = /3 1 , /3 2 , K = (3 2 . The functions 
p and M are assumed to be continuous and the functions bi,di and fii are of class C . 
We assume 

ri = bi-di>0, i e {1,2}. 

4) Similarly to (13. 16ft . the interaction rates between cells satisfy 

^- i,j €{1,2}. (3.17) 



Remark that Assumption (H2) 1) means that at a large scale K, the individuals are 
influenced in their ecological behavior by the cells if the number of the latter is of order K± 
for cells of type 1, resp. of order K 2 for cells of type 2. On the other side the hypothesis 
(H2) 2) may be a consequence of a fixed amount of available resources to be partitioned 
among all the individuals. Larger systems are made up of smaller interacting individuals 
whose biomass is scaled by 1/K, which implies that the interaction effect of the global 
population on a focal individual is of order 1. 

Examples 

(i) If K\ = K 2 and if the individual rates B K , D K , ct K only depend on x,ni,n 2 by the 
proportion of cells of type 1, then (|3.15p is satisfied. 

(ii) Assume that K± = K 2 = K and that the functions B K ,D K ,a K only depend on the 
weighted total number of cells ^(n± + n 2 ). 
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3.1 A convergence theorem 

We assume that the sequence of random initial conditions Y Q K converges in law to some 
finite measure vq 6 Mp(X x R + x R + ) when k — > oo. Our aim is to study the limiting 
behavior of the processes Y K as k — > oo. 

The generator L K of (Y t K )t>o is easily obtained by computing, for any measurable function 
4> from M F (X x R + x R + )~into R and any /i G M F (Af x M + x R + ), 

L>(^) = a^(0(y, K )) 4=o . 

In particular, similarly as in Theorem 12,61 we may summarize the moment and martingale 
properties of Y K . 



Proposition 3.1 Assume that for some p > 3, E((Yq% 1) p + (Y Q K ,yl + < Th- 



en 



(1) For any T > 0, E (sup te[0)T] (Yf, l)f + su Pte[0iT] <Y t K , yf + y 2 2 )j < +00 . 

(2) For am/ measurable bounded functions f,gi,g 2 , the process 

M^ 9 = (Y t K , fg x g 2 ) - (Y K , f 9l g 2 ) -If I (b k (x, K m ,K 2 y 2 )(l - p{x)) 

JO JXxR\ I v 

- {D K {x,Kiyi,K 2 y 2 ) + a K {x,Kiyi,K 2 y2) U *Y s K (x,y 1 ,y 2 ))j f(x)gi(yi)g 2 (y 2 ) 
+ p(x)B K (x, Kiy u K 2 y 2 ) j f(x + z)g 1 (y 1 )g 2 (y 2 )M(x, z)dz 

+ f(x)(gi(yi + -^-) - gi (yi)) gi {yi) h{x) K x y x 
+ f(x)gx(y\){g 2 {y 2 + — ) - 32(2/2)) h(x) K 2 y 2 
+ f(x){gi(yi - -^-) - gi{.yi))g2(y2)(d 1 (x) + Pi{x)(\ n yi + Ai 2 y 2 )J-^iyi 

+ f{x)gx(y x ){g 2 {yi - -^-) - 32(2/2)) (tfcfc) + /^OO^iyi + A 22 y2))^2y2 j 

Y s K (dx,d yi ,dy 2 ) ds (3.18) 
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is a cddldg square integrable martingale starting from with quadratic variation 



(M 



Kjg )t = — 
' K 



1 * 



JXxM? + 



B K (x,K 1 y 1 ,K 2 y2)(l ~ P( x )) 



+ (D K (x, K lVl , K 2 y 2 ) + a K (x, K lVl , K 2 y 2 ) U * Y s K (x, y u y 2 )) ) f 2 (x)g 2 {yi)gl{y 2 ) 
+ p(x)B K (x, K iyi ,K 2 y 2 ) f 2 {x + z)gj{ yi )g1(y 2 )M(x, z)dz 



+ f 2 (x) (31(31 + -^-) - gi(yi)) 2 g\(y2) h(x) K x y x 

+ f 2 (x) gf(yi) (52(2/2 + -£r) - 32(2/2)) 2 b 2 (x) K 2 y 2 

+ f 2 (x) (31(2/1 - -j^r) - 3i (yi)) 2 32(32) (^i(x) + /3i(x)(Anyi + Ai 2 y 2 ))^iyi 

+ / 2 (^) 9? (2/l) (32(32 - - 32(32)) 2 (^0*0 +^2(s)(A2l2/l + A 2 232))-^22/2 



Y s K (dx,d yi ,dy 2 ) ds. (3.19) 



We can now state our convergence result. 



Theorem 3.2 Assume (H2). Assume moreover that the sequence of initial conditions 
y K € Mp[X x R+) safe/ies sup K E((y o K , l) 3 ) < +00 and sup K E((y o K , yf + yf)) < +00. // 
Yq converges in law, as k tends to infinity, to a finite deterministic measure vq, then the 
sequence of processes (^t K )o<t<T converges in law in the Skorohod space B([0, T], Mp(X x 
1Lj_)), as k goes to infinity, to the unique (deterministic) measure-valued flow v £ C([0,T], Mp{X x 
M+)) satisfying for any bounded and continuous function f and any bounded functions gi,g 2 
of class Cl, 



(vt,fgi92) = (vo,fgm)+ / \ [B(x,y 1 ,y 2 )(l - p(x)) 

JO JXxR\ I v 

- (D(x,yx,y 2 ) + a(x,yi,y 2 ) U *v s (x,y 1 ,y 2 ))\f(x)g 1 (yi)g 2 (y 2 ) 
+ p(x)B(x,yx,y 2 ) Jf(x + z)M(x, z)dz 31(31)32(32) 
+ f{x) 31(2/1)32 (32)^1 (x) yi + gx{yx)g' 2 {y2)b 2 {x) y 2 

- 31(31)32(32) (dx (x) +/3i(x)(An3i + Ai23 2 ))yi 

- 3i (31)32(32) (d 2 (x) + /3 2 (x)(X 21 y 1 + A2232)) 32 }v s (dx,dyx,dy 2 ) ds. (3.20) 



Note that for this dynamics, a transport term appears at the level of cells. 



Remark 3.3 • A solution of (|3,20p is a measure- valued solution of the nonlinear 
integro-differential equation 

—v t = (B(l-p)-(D + aU*v t y)vt + (Bpvt)*M-Vy(cvt) (3.21) 



13 



with 



ci(x,y) ■= yi(n(x) - /3 1 (x)(\ u y 1 + \ 12 y 2 )) 

C2(x,y) := m {r 2 {x) - fo{x){\ 2 iyi + A222/2)) • (3.22) 

Thus, the existence of a weak solution for Equation (|3,2ip is obtained as corollary of 
Theorem \3Z 



We deduce from (|3,20p the limiting dynamics of the total number of individuals: 



/ / [B(x,yi,y 2 ) - D(x,y 1 ,y 2 ) 
Jo JxxRl v 



(v t ,l) = (u ,l) + 



while the total number {vt,yi) of cells of type i at time t is obtained by taking 
/ = l, gi (y) = y,g, = 1 (i ^ j) in ^M) ■ 



a(x,y 1 ,y 2 ) U * v s (x,yi,y 2 ) )v s (dx, dyi, dy 2 ) ds, (3.23) 



(vt,yi) = (vo,yi) 

+ / / [B(x,y 1 ,y 2 ) - D(x,y l7 y 2 ) - a(x,yi,y 2 ) U *v s (x,y 1 ,y 2 ))y i v s (dx,dyi,dy 2 )ds 
Jo JxxR 2 + v 7 

+ / / [(bi(x) - di(x))yi- f3i(x)(\uyi + \ ij y j )y i )v s (dx,dy 1 ,dy 2 ) ds. (3.24) 

'0 JA'xio' 2 v 



Proof The proof of the theorem is obtained by a standard compactness-uniqueness 
result (see e.g. [3]). The compactness is a consequence, using Prokhorov's Theorem, of the 
uniform tightness of the sequence of laws of (Y t K ,t > 0). This uniform tightness derives 
from uniform moment estimates. Their proof is standard and we refer for details to |10) . [7] 
Theorem 5.3 or to [2j. To identify the limit, we first remark using (|3.19p that the quadratic 
variation tends to when K tends to infinity. Thus the limiting values are deterministic 
and it remains to prove the convergence of the drift term in (I3.18P to the one in (|3.20p . The 
drift term in (|3.18p has the form Jq(Y s k , A K (Y s K )(f gig 2 ))ds and the limiting term in (|3.20p 
has the form J" * (v s , A(v s ){f g\g 2 ))ds. (The exact values of A K and A are immediately given 
by |ngD and (13301) ). 

Thus, let us show that if Y K is a sequence of random measure- valued processes weakly 
converging to a measure- valued flow Y and satisfying the moment assumptions 

supE(sup(Y t K , l) 3 ) + supE(sup(Y t K ,y 2 )) < +00, (3.25) 

then (Y t K , A K (Y t K )(f gig 2 )) converges in L 1 to (Y t , A(Y t )(f g±g 2 )) uniformly in time t G [0, T]. 
We write 

{Y «,A«(Y t «)(f 9l g 2 )) - (Y t ,A(Y t )(f 9l g 2 )) 

= (Y«,A«(Y«)(fg m ) - A(Yn(f9i92)) + {Yf, A(Y«)(fg ig2 ) - A(Y t )(fg l92 )) 
+ (Y t K -Y u A(Y t )(f gi g 2 )). (3.26) 
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The convergence of the first term to zero follows from Assumptions (H2) and (|3,25|) and 
from the following remark, that for C^-functions g\ and 52, the terms 



Ki(gi(yi - —) - + gi(yi 



converge to in a bounded pointwise sense, which allows us to apply the Lebesgue's 
theorem. 

The convergence of the second term to is immediately obtained by use of (|3.25p . since 
the functions a and U are continuous and bounded. 

The convergence of the third term of (|3,26p is due to the weak convergence of Y K to Y. 
We know that for all bounded and continuous functions (ft, the quantity (Y t K — Yt,4>) tends 
to 0. The function A(Yt)(fgi92) is a continuous function which is not bounded because 
of linear terms in y and y 2 . Thus we need to cutoff at a level M replacing y by y A M. 
The remaining terms are proved to go to using (|3.25p . Hence we have proved that each 
limiting value satisfies (I3.20p . 

We have now to prove the uniqueness of the solutions v G C([0, T], Mp{X x of 
(|3.20p . Our argument is based on properties of Lotka-Volterra's flows. Firstly we need the 
following comparison lemma. 

Lemma 3.4 If ut is a non negative function with positive initial value and satisfying for 
some a,b G Ml the inequality 



d 

Vt > 0, — ut < au t - bu t , 



2 



—S t > (a-b(Ut + ut))St, S 



0. 



then < supn t =: u < +00. 

t>o 

Moreover is an absorbing value: if Ut = then for all t > to, ut = 0. 
Proof of Lemma 13.41 Let us define Ut as solution of the associated logistic equation 

^£ = allt - bllt, U = u - 

Then §t(U t - u t ) > a(U t - u t ) - b{Uf - uj). With 5 t := U t - u t it holds 

d_ 
dt 

Let us show that t 1— > St increases, and therefore is positive. For t = 0, since So = 

0, ^f\t=o > 0. Thus S t > in a neighborhood of 0. 

Let define to := sup{t > : St = 0}. If to = +°° the problem is solved. 

If not, Ut = Ut on [0, to]. Let us now define t\ := inf{t > to : St < 0}. If t\ = +00 the 

problem is solved. If t\ < +00, by continuity 5^ = and then ^1^=^ > 0. Thus, in a 

small time intervall after t\, St would increase and be positive, which is a contradiction 

with the definition of t\. Therefore t ^ St increases and stays positive, which implies that 

< u := sup ut < sup Ut < +00. 

t>0 t>0 

□ 

Let us now recall some properties of the Lotka-Volterra's flow involved in the cell dynamics. 
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Lemma 3.5 Let £q € [0,T], x S X and y = (2/1,2/2) £ ^+ be given. The differential 
equation 

^y(t) = c(x, y(t)), t e [t Q , T], with y(t ) = y (3.27) 

where c defined in f|3.22[) . admits in a unique solution 1 1— > tp$ ,y (t) = (<P^i(t), ^"^(i)). 
Moreover the mapping (x, t, s, y) 1— > (p x ' V (t) is C° in x G X and C°° in t,s,y £ [0, T] 2 x M.+ 
and is a characteristic flow in the sense that for all s, t, u, 

<fi2 V (t) = where z = <p'»(u). (3.28) 

Proof of Lemma 13.51 Since the coefficients c, are of class C 1 and thus locally bounded 
with locally bounded derivatives, the lemma is standard (cf pQ) as soon as the solution 
does not explode in finite time. The latter is obvious, since the quadratic terms are non 
positive. Indeed, the functions (2/1,2/2) are dominated by the solution (z±, z 2 ) of the system 

d 

— zi(t) = ri(x)zi(t) - j3i{x)\uzf ; Zi(0) = y i} i = 1, 2, 

and we use Lemma 13.41 
The flow clearly satisfies 



^f(t) = y 1 exv^(r 1 (x)-(3 1 (x)[\nv%f(s) + \ 12 <p t °$(s))ds 
= 2/ 2 exp ( f(r 2 (x) - p 2 (x){\ 21l p%?(s) + \ 22 ^' 2 % \s))ds 



to 



□ 



The proof of uniqueness will be based on the mild equation satisfied by any solution of 
(I3.20j) . Let us consider a function G defined on X x of class C 1 on the two last variables 
and for any x £ X, let us define the first-order differential operator 

dG dG 
CG(x, y) := c^x, y) — (x, y) + c 2 [x, y) — (x, y) = c • V y G (x, y), (3.29) 
dyi dy 2 

where the notation • means the scalar product in M 2 . 
Then the function G(s,t, x,y) := G(x,(fx V {t)) satisfies 

|g = y y G(x,^(t))-^ v (t) 

= c(x,^y(t)).^7 y G(x,^ y (t)) 

= CG(x,ip s ^ y {t)) = CG (3.30) 

Let us fix t > 0. We deduce from (|3.30p and from the flow property (|3.28p that G satisfies 
the backward transport equation: 

d ~ 

— G + £G = 0, Vs<t with G(t,t,x,y) = G(x,y). (3.31) 
as 
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We now write (|3.2Up applying the measure Vt to the time-dependent function (s, x, y) \— )■ 
G(s,t,x,y) where G(x,y) = f(x)g(y) and obtain the mild equation 



(vtjg) = (v Jg°<P° x ' y (t))+ / \(B(x, yi ,y 2 )(l-p(x)) 

JO JXxRl I v 

- {D(x,y 1 ,y 2 ) + a(x,y 1 ,y 2 ) U * v s (x, y x , y 2 ))j f(x)g o <p s x ' y (t) 
+ p{x)B(x,y 1 ,y 2 ) J f{x + z)g o ^ s J y rZ {t)M{x,z)dz\v s {dx,dy 1 ,dy 2 ) ds. (3.32) 

(The last term involving the quantity -^g o ip x ' y (t) + Cg(ifx y (t)) vanishes by (13.31 1) . ) 



Let us now consider two continuous functions v and v in C([0, T], Mp{X x solutions 
of (|3.2Up with the same initial condition vq. Then the difference of both solutions satisfies 

j-t 

(vt - v t ,fgig 2 



'0 JxxR: 

+ p(x)B(x, yi,y 2 ) f(x + z)g o (p s x f z (t)M(x, z)dz 



B(x, yi,y 2 )(l - p(x)) - D(x, yi,y 2 )j f(x)g o ^(t) 
v s (dx,dyi,dy 2 ) - v s (dx,dy 1 ,dy 2 )) 
a(x,y 1 ,y 2 )f(x)go(p s x > y (t)(U*v s (x,y 1 ,y 2 )v s (dx,dy 1 ,dy 2 ) - U * v s (x,y 1 ,y 2 )v s (dx,dyi,dy 2 ] 



The finite variation norm of a measure v is defined as usual by 

IMIfv := svp{{v, h),h measurable and bounded by 1}. 

Since all coefficients are bounded as well as the total masses of i>t and Vt , it is easy to 
show that there exists a constant Ct such that 

\\vt - v t \\ FV < C T \\v s - v s \\ FV ds, 



Jo 

which implies, by Gronwall's Lemma, that v and v are equal . □ 

Let us now prove that if the initial measure has a density with respect to Lebesgue measure, 
then there exists a unique function solution of (|3,2ip . That gives a general existence and 
uniqueness result for such nontrivial equations with nonlinear reaction and transport terms, 
and a nonlocal term involved by the mutation kernel. The existence takes place in a very 
general set of Z^-functions. 

Proposition 3.6 Assume that the initial measure vq admits a density <pQ with respect to 
the Lebesgue measure dxdy\dy 2 ; then for each t > 0, the measure vt solution of (I3.2ip also 
admits a density. 

Proof Let us come back to the equation (|3.32p satisfied by v. Using basic results on linear 
parabolic equations, we construct by induction a sequence of functions (4> n ) n satisfying in 
a weak sense the following semi-implicit scheme: 4>q +1 = 4>o an d 
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r 1 . 



fg) = (<h,fg°<p% v (t))+ / / N (b{x, Vu y 2 ){i- P {x))f(x)go^y(t) 



- [D(x,yi, y 2 ) + a(x, yi, y 2 ) U * (j% (x,y 1: y 2 ))jf(x)g o ^ ,s/ (t)^ +1 (z, 2/1,2/2) dxdy 1 dy 2 ds. 

(3.33) 

Thanks to the nonnegativity of 0o and of the parameters .B, p, 1 — p, and applying the 
maximum principle for transport equations (Cf. [I]), we can show that the functions (p n 
are nonnegative. Taking / = g = 1 and thanks to the nonnegativity of the functions (j) n 

and to the boundedness of the coefficients we get 

sup||C +1 Hi < ll^olli + Ci I sup||C||id«, 

s<t JO u<s 

where the constant C\ does not depend on n and can be chosen uniformly on [0,T]. By 
Gronwall's Lemma, we conclude that 

supsup||^||i < Uohe ClT . (3.34) 

n t<T 

Let us now prove the convergence of the sequence (jf 1 in L°°([0, T], L 1 ). A straightforward 
computation using f|3.33[) , (|3.34p , the assumptions on the coefficients and similar arguments 
as above yields 

sup - Clli < C 2 f (sup nc +1 - Clli + ™ P ui - €r%)ds, 



s<t JO v u<s u<s 

where C 2 is a positive constant independent of n and t £ [0, T]. It follows from Gronwall's 
Lemma that for each t < T and n, 

,"- J - r* / „^iun_^-l| 



sup||C +i -Clli<C 3 / sup ||C -C^IMs- 

We conclude that the series Yl n sup tg [ ,T] ll^™" 1 " 1 ~~ converges for any T > 0. Therefore 
the sequence of functions {(j) n ) n converges in L°°([0, T], L 1 ) to a continuous function 1 1-^ <pt 
satisfying 

sup ||0t||i < ||^o||ie ClT . 

t<T 

Moreover, since the sequence converges in L , the limiting measure <j)i{x,y\,y 2 )dxdy\dy 2 
is solution of (|3.32p and then it is its unique solution. Hence, that implies that for all t, 

v t (dx,dyi,dy 2 ) = 4> t {x, y 1 , y 2 ) dxdyidy 2 . 

□ 

We have thus proved that the nonlinear integro-differential equation (|3.2ip admits a unique 
weak function-valued solution as soon as the initial condition <po is a L 1 -function, without 
any additional regularity assumption. 
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3.2 Stationary states under a mean field assumption and without trait 
mutation 

This part is a first step in the research of stationary states for the deterministic measure- 
valued process (vt,t > 0) defined above. We firstly remark that equation f|3.23|) . which 
determines the evolution of the total number of individuals t t— >■ (i>t,l), is not closed if 
the functions U, B,D or a are not constant, which makes the problem very hard. In 
this section we consider the simplest case where the individual ecological parameters B 
and D and the cell ecological parameters b% and d% are constant and where the mutation 
probability p vanishes. Moreover, we work under the mean field assumption, that is the 
competition/selection kernel U is a constant. We consider two different cases corresponding 
to different selection rates a. 



3.2.1 Case with constant selection rate 

Let us assume that the selection rate a is constant. In this case, the mass equation (13.23|) 
is closed and reduces to the standard logistic equation 

(v t , 1) = (v , 1) + f {v., 1) ({B — D) — aU(v s , 1)) ds, (3.35) 
J o 

whose asymptotical behavior is well known: the mass of any stationary measure Voo satisfies 

(B-D)(v 00 ,l)=aU(v 00 ,l) 2 - 
Either R := B — D < and there is extinction of the population, that is 

lim (v t , 1) = (voo, 1) = 0. 

t— >+oo 

Or R > and the mass of the population converges to a non degenerate value 



lim (v t ,l) = (voo,!) = 

t— >+oo 




(3.36) 



Furthermore, the convergence of the mass holds exponentially fast: due to (|3.35p . 

d 

Wlivt - V^, 1) = -aU(v t , l)(v t - Voo, 1)- 



Thus {vt -Voo,l) = (wo - Woo, 1) e~ aU ^) ^ (3 37) 

which vanishes exponentially fast. 

Assume R > in such a way that the mass of the population does not vanish. In what 
follows we will need the following notations: 

(v, 1) := sup(v t , 1) < +oo 
t 

and 

a := sup at(< +oo) where at := R — aU(v t , 1) = —all(vt — VooA) ■ 
t 
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Let us now consider the weak convergence of the measures Vt towards the stationary mea- 
sure Voo, which is concentrated on the equilibrium state of the Lotka-Volterra dynamics. 
Applying equation (I3.20P to any bounded smooth fonction g(y) = gi(yi)g 2 (y 2 ), 

d . . . . . dg dg 

-^(vt,g) = a t (v t ,g) + {vt,n yi — + r 2 y 2 —) 

-{vt,Pi-7r-(Miyi + Ai 2 y2)yi + fo^-i^iyi + X 22 y 2 )y 2 ) 
dyi ' dy 2 y ' 

= a t (v t , g) + {v t ,Cg) (3.38) 

where the differential first order operator C = c ■ V is the same as in (|3.29p but without 
dependence on the trait x. Using the flow of Lotka-Volterra equation (see (|3.27p ). we 
represent the mild solution of ()3.38j) as 

(vt,9)= I go<p°> v (t)vo(dy)+ fa. f go^y(t)v s {dy)ds. (3.39) 
Jw? + Jo 

Let us firstly recall the long time behavior of the Lotka-Volterra system (|3.27p in case 
where the coefficients a don't depend on x(see Istas [8j). 

Lemma 3.7 Any solution of 

J^i(i) = Ift(t)(ri-^i(Anyi(t)+Ai2ite(t))) 

^ya(t) = ite(t)(r2-A(A 2 iyi(t)+A 2 2ite(t))) (3.40) 

with non-zero initial condition in converges for t large to a finite limit, called equilibrium 
and denoted by n = (7Ti,7T2) G \ {(0,0)}. It takes the following values: 

1 - 71 = (ftTIT' ) r 2 \u - ri\ 2 i < (resp. = and ri\ 2 2 - r 2 \u > 0). 

2. tt = (0, #jx^) if n\ 22 - r 2 X 12 < (resp. = and r 2 X u - r x X 21 > ). 

3. If r 2 Xu - riAi2 > and n.A 22 - r 2 A 2 i > 

_ _ / /3iAi 2 (6 2 - d 2 ) - f3 2 X 22 (h - di) /3 2 A 2 i(6i - d{) - /3iAn(6 2 - d 2 ) \ ^ ^ 
V /?i/3 2 (Ai2A2i — A11A22) /3i^ 2 (Ai 2 A 2 i — A11A22) ' 

Therefore we obtain the following convergence result. 

Proposition 3.8 The deterministic measure-valued process vt converges for large time t - 
in the weak topology - towards the singular measure concentrated on the equilibrium state 
7r of the associated Lotka- Volterra dynamics: 



_R 

1 all 



lim v t = — Sfaw), 



where it = (711,^2) is defined in Lemma\3_ 
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Proof First, the Lotka-Volterra flow ip°' y (t) converges for large t towards (7Ti,7T 2 ) given 
by Lemma 13.71 Since the test function g is continuous and bounded and v o has a finite 
mass, Lebesgue's dominated theorem implies that the first term in the right hand side of 
(|3.39p converges: 



lim 

t 



g o (p°' v (t) v (dy) = / limgo(p°' y (t)v Q (dy) = 5(711, 7T 2 ) (t>o, 1)- 



Secondly, as already seen in (|3.37p . the mass (ft, 1) of the total population converges 
exponentially fast to its equilibrium size, that is at converges exponentially fast to 0: 

3c > 0, 3t , Vs > t a s < e~ cs . 

Therefore the second term in the right hand side of (|3.39p can be disintegrated, for t larger 
than to, in the sum of two integrals over [0,to] and [to,t]. The control of the integral over 
[to,t] is simple: 



o,, 



t Jm. 2 , 



go<p s ' y (t)v s (dy) ds 



< (v,l)sup|g(y)| / e cs ds 
y Jt 



which is as small as one wants, when to is large enough. 

On the compact time interval [0, to] the following convergence holds: 



lim 

t 



to 



&s \ g° (f s ' y (t)v s (dy)ds 



to 



lim g o (f s,y (t) v s (dy) ds 



t 



9(^1,^2) / «s / v s {dy)ds. 



Therefore for large time t > to, (vt,g) is as close as one wants to 



sOi,7r 2 ) (« ,1) +ff(7Ti,7r 2 ) 



to 



v s (dy) ds = 5(71-1, 7r 2 ) {v to ,l). 



For to large enough, this last quantity is close to 5(71-1, 7r 2 ) (voo, 1} = ^ (5r^ li7r2 \,g). 
This completes the proof of the weak convergence of the measures vt ■ 



□ 



Remark 3.9 The stationary state is a singular one even if the initial measure vo has a 
density: the absolute continuity property of the measure vt is conserved for any finite time 
t, but it is lost in infinite time. 

Convergence of the number of cells 

First we prove the boundedness of the number of cells of each type and the boundedness 
of its second moment. To this aim, we compare the multitype dynamics with a dynamics 
where the different types do not interact, which corresponds to two independent monotype 
systems. 

Lemma 3.10 // (vq, 1) + {vo,yf) < +00 then sup t > (v t ,yf) < +00. 
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Proof Let us firstly prove that sup <>0 {vt,Ui) < +00. 

At time t = , («o>2/i) < i v 0, 1) + (^0)2/?) < +00. Moreover, equation (|3.24p reads now 


0^(vt,yi) = [R- aU(v t ,l))(v t ,yi) + (h - di)(v t ,yi) - Pi(Xu(v t ,y 2 ) + hj(vt,yiyj)) 

< (a t +bi- di)(v t ,yi) - (3i\u{v t ,yf) 

< (a t + bi- di)(v t ,yi) - ' " (v t ,yi) 2 

< ( a + r t )(v t , yi )-^-(vt,y t ) 2 . (3.42) 

This inequality is a logistic one in the sense of Lemma 13.41 Therefore one deduces that the 
number of cells of type i is uniformly bounded in time: 

sup (ut, 2ft) < +00, i = 1,2. 



By (|3.20p applied with / = 1, 51(2/1) = 2/?> 02 = 1> one obtains 
< (at + 2n)(i; t ,2/f)-2^Aii(^,2/?) 



:i v t,yl) = a t {vt,y1) + 2ri{v t ,yl) - 2(3i^\ n (v t ,yf) + X 12 (v t ,y 2 y2) 



< (a t + 2r 1 )(vt,y 2 )-2p l \ ll —!—(vt,yj) 2 

(vt,yi) 

< (a + 2n ) (v t , yl) - 2ft An — ^- (t* , 2/? ) 2 

(«,yi) 



since 



(v t ,yl) 2 < (v t ,yf}(vt,yi). 
This inequality on (vt,y 2 ) is of logistic type as (|3.42p . Lemma 13.41 implies 



(v,yf) := sup(v t ,yf) < +00. 

i>0 



The same holds for (v, 2/f)' ^— ' 

Proposition 3.11 // (vo,yi) < +00 and (vo,yf) < +00, i/ien i/ie foiaZ number of cells of 
each type per individual fe^r stabilizes for t large: 

,. (^,2/i) 

lim -j = 7Tj. 

Proof Due to Proposition I3.8[ the family of measures (w^)t converge weakly towards Uqo. 
Moreover, by Lemma [3. 101 the second moments of Vt are uniformly bounded. Therefore 2ft 
is uniformly integrable under the family of (vt)t which leads to : 

lim (f t ,2ft) = (lim v u yi) = («<»,&). 

t— >+oo t— >+oo 

□ 

Let us underline the decorrelation at infinity between cell and individual dynamics. 
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3.2.2 Case with linear selection rate 

Suppose now that the selection rate a does not depend on the trait x but is linear as 
function of the number of cells of each type : 

3ai,o 2 e]0, 1[, a(x,yi,y 2 ) = ot\y\ + a 2 y2 =■ a-y. 

With other words the selection increases linearly when the number of cells increases. 
The new main difficulty comes from the fact that the mass equation is no more closed : 

(v t , 1) = (v , 1} + / {v., 1) [R - U(v s ,a ■ y)) ds, (3.43) 
J o 

which has as (implicit) solution 

(t*,l) = (v ,l)e-ti {u(vs,a-y)-R)ds ^.M) 

For this reason, unfortunately, we did not succeed in proving the convergence in time of 
(vt, 1). Nevertheless, we can conjecture some limiting behavior of the process. 

Conjecture : The deterministic measure-valued process vt converges for large time t 
towards the following stationary value 

R 

lim v t = Vco := — ■ 5 {7T } , (3.45) 

where tt = (ttj,^) is given in Lemma 13.71 

In this case too, the asymptotic proportions of the cells of different types per individual 
would become deterministic and independent. 

Some partial answers 

• Equation (13.43|) implies that any stationary measure Voo should satisfy 

(uoa,l>(^-^(«oo,a-y» =o. 

Then, either {v^, 1) = 0, that means the extinction of the individual population 
holds, or 

(voo, a ■ y) = (voo, aiyx + a 2 y2) = —JJ— = JJ ( 3 - 46 ) 

which describes a constraint between the limiting number of the different types of 
cells. 

• Boundedness of the number of cells. 
Lemma 3.12 

(vo,yi) < +oo => sup (v t ,yi) < +oo. 

t>0 
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Proof The number of cells of type i satisfies 

d_ 
at 



jr,(vt,yi) = (v t ,yi)(R + ri) - U(v t ,yia ■ y)(v t , 1) - Pi(Xu(v t ,yf) + \j{vt,yiyj)) 



< (R + ri)(v t ,yi) -aiU{vt, yi ) 2 (3.47) 
which reduces to the monotype case solved in Lemma l3.4i □ 

Identification of a unique possible non trivial equilibrium. 

Applying Equation (|3.2U|) to / = 1, gi(y) = e~ ZiV and letting t tend to infinity, we 
remark that the Laplace transform Loo(z) of any non vanishing stationary state Voq 
should satisfy 

RL^z) - U( Voo , l)( Voo , (a • y)e~ z - y ) 

2 

~ Yl Zl ( r i( v oo, yie~ z ' y ) - Pi(\i{voo,yh~ z ' v ) + \j(voo,yiyje~ z ' y })) = o 
i=i 

RL^ + UL x (0)( ai ^ + a 2 ^) 

OZ\ OZ2 

1=1 1 J 

with usual boundary conditions 

Loo(0) = (Voo, 1), -q^-(0) = -{Voo^i). 

The unique non trivial solution of this p.d.e. is 

L 00 (z) = (v oc ,l)e- jr - z , 

where (^,1) = and where 7Tj, the equilibrium proportion of cells of type i in 
the global population, has to be equal to the equilibrium proportion given in Lemma 

ETT1 7T = 7T. 

Local stability of the non trivial equilibrium Voo '■= 

Although we cannot control the convergence of (v-t, 1) to a positive number, we can 
analyze the stability of the nontrivial stationary state in the following sense. 

Stability of the mass around its positive stationary value ^ 

Let start with vq = Vqo + £<5(£ ll f 2 ) 3 where e is small and (Ci: C2) £ From the mass 
equation f|3.43[) one obtains for t small : 

J^,l> = (v t ,l)(R-U(v t ,a-y)) 

~ ((v 00 ,l) + e)(R-U{v 00 ,a-y) -eUa-C) 



~ — e hoe). 

a ■ 7r 
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This quantity is negative for small e, which implies the stability of the mass around 
its positive stationary value. 

Stability of the number of cells of each type around its limit value i/max(n,r2) < R 
We prove it only for the type 1. From (|3.24p we get an expansion in e of the variation 
of the global number of cells of type 1 for small time : 

d 

0^(vt,yi) = {vt,yi){R + n) -U(v t ,yia-y)(v t ,l) - h (An («t,y?) + Ai 2 (wt, J/12/2)) 

- ((«oo, + eCi) (P + n) - ((woo, 2/1 a • y) + eCi a • C) (Ko, 1) + e) 
(An («oo,Z/i) + Ai 2 (*Uoo, 2/12/2}) - £/3i(AnCi 2 + A12C1C2) 

= e((i? + ri)Ci -^(voo,2/ia-y> -t/Cia-C(^oo,l) - £l(AnCi + A12C1C2)) + o(e) 
= A(Ci,C 2 )+o(e) 

where -Pi (2/1, 2/2) < Pi(yi) f° r ah 2/2 > 0, with 

Pl(X) := -(C/a!^, 1) + /3iAu)X 2 + (i? + n)X - Ua^v^, 1). 

As second degree polynomial Pi is negative if its discriminant is non positive. This 
condition is fulfilled when 

(R + nf - 4i7ai7rf (woo, 1) {Ua x ( Voo , 1) + ft Ail) < 0. 
It is true as soon as 

(R + ri) 2 -4R 2 <0^n<R. 

Thus if max(ri, r2) < R, the number of cells of each type is stable around its limiting 
value. 

4 Diffusion and superprocess approximations 

As in the above section we introduce the renormalization k = (K, K\,K<i) both for in- 
dividuals and for cells. Moreover we introduce an acceleration of individual births and 
deaths with a factor K* 1 (and a mutation kernel Mk with amplitude of order K^l 2 ) and 
an acceleration of cell births and deaths with a factor K\ (resp. K<i). 

We summarize below the assumptions we need on the model and which will be considered 
in all this section. 
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Assumptions (H3): 

1) There exist continuous functions T,B,D,a on X x R + x R + such that 

B K (x, ni ,n 2 ) = T(x, ^) + B(x, ^); 

D K (x, m, n 2 ) = IT 7 r(x, — -^-) + -|-); 

ill A 2 Ai A2 

n l n 2 

a K (x,ni,n 2 ) = a(x, — , — ). (4.49) 

The function T is assumed to be bounded and B,D,a satisfy Assumptions (HI). 

2) As before, the competition kernel satisfies 

where U is a continuous function which satisfies Assumption (HI). 

3) The mutation law z 1— > Mk(x, z) is a centered probability density on X — x. Its covariance 
matrix is "ffi Id, where a is a continuous function. We also assume that 



lim K v sup / \z\ 3 M K (x,z)dz = 0. 

K-*oo x J 



The parameter p K stays unchanged: p K (x) = p{x) . 

4) At the cell level, we introduce Lipschitz continuous functions bi,di on X and a continuous 
function 7 such that 

b itK (x) = Kij(x) + bi(x); 

d i , K {x) = K il {x) + d i {x), i = l,2. (4.50) 
The interaction between the cells is rescaled according on their type : 

A& ^ «,i€{l,2}. (4.51) 

The other parameters stay unchanged: fi\ K = f3\, /3 2)/{ = /3 2 . 

5) Ellipticity: The functions p, a , 7 and T are lower bounded by positive constants and 
a y/pT and are Lipschitz continuous. 

As in the section 3, we define the measure-valued Markov process (Z^)t>o as 



T s / j u n\ (t) m (t) ■ 
We may summarize as in Proposition 13.11 the moment and martingale properties of Z K . 
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ft 

lO JXxR 2 , 




Proposition 4.1 Assume that for some p>3, E((Zft, l) p + (Zft, y\ + y|)) < +00. Then 

(1) For any T > 0, E (sup t6[0)T] (Zf , l) 3 + sup te[0jT] (Zf , y\ + y|)) < +00. 

(2) For any measurable bounded functions f,gi,g2, the process 

M t Kj9 = (Z?,fg m )-(Z5,fg ig2 ) 

B{x,yi,y 2 ) - D(x,y 1 ,y 2 ) - a(x,y 1 ,y 2 ) U * Z^(x,y 1 ,y 2 )^j f(x)g 1 (y 1 )g 2 (y 2 ) 

+ p(x) (K" T(x, yi ,y 2 ) + B(x, Vl ,y 2 )) J (f(x + z) - f{x)) gi { yi )g 2 {y 2 )M K {x, z)dz 

+ f(x){gi(yi + ^-) - 91 (2/1)) 52^2) {K^[x) + h{x)) K lVl 

+ f(x)g 1 (y 1 )(g 2 (y 2 + — ) - 92(1/2)) (^27^) + ^(x)) K 2 y 2 

+ f(x)(gi(yi - -^-) - 91 (2/1)) 52 (92) (^n(^) + + Pi{x){yi\u + y2^i2))K iyi 

+ f(x)gi(yi)(g 2 (yi - -^-) -92(2/2)) (#270*0 + d 2 ( a + /?2 0*0 (2/1^21 + 2/2A22)) # 2 y 2 j 

Z*(dx,d yi ,dy 2 ) ds (4.52) 
is a cadlag square integrable (F t )t>Q-martingale with quadratic variation 

(M">/9) t = l T / {(2^r(x,y 1 ,y 2 ) + B(x,y 1 ,y 2 ) 

+ D(x,yi,y 2 ) + a(x,y 1 ,y 2 ) U * Zg(x, yi ,y 2 ^j f 2 (x)gl(y 1 )gj(y 2 ) 
+ p(x) (K v T(x, yi ,y 2 ) + B(x, yi , y 2 )) f {f(x + z) - f(x)) 2 M K (x, z)dz g 2 (yi)g 2 2 (y 2 ) 
+ f(x) (91 (yi + ^-) - 91 (9i)) 2 91(2/2) (#170*0 + 61 (a:)) #i2/i 
+ fix) 9i(yi) (52(92 + -^-) - 92(92)) 2 (#270*0 + (»)) #292 

+ /V) (91 (91 - ^-) - ffi(2/i)) 2 91(2/2) (#i7(z) + di(x) + /3i(x)(yiAn + y 2 \i 2 ])K iyi 

+ / 2 0*0 ffi(yi) (92(92 - -^-) - 92(92)) 2 (#270*0 + d2(x) + ^ 2 (x)(yiA 2 i + 92A22))#292 

Z«(dx,d yi ,dy 2 ) ds. (4.53) 

We assume that the sequence of initial conditions Zq converges in law to some finite 
measure Co- Let us study the limiting behavior of the processes Z K as k tends to infinity. 
It depends on the value of i] and leads to two different convergence results. 
As before we denote by r$ the rate bi — d{. 
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Theorem 4.2 Assume (H3) andrj s]0, 1[; suppose that the initial conditions Zq E Mp{X x 
M+) satisfies sup K E((Zq , l) 3 ) < +00. If further, the sequence of measures (Zq) k converges 
in law to a finite deterministic measure Wq, then the sequence of processes (Z t K )o<t<T con- 
verges in law in the Skorohod space O([0, T], Mf(X x K+)), as k goes to infinity, to the 
unique (deterministic) flow of functions w G C([0,T],L 1 (Af x R+)) weaA; solution of 



— w t = [B-D-a U *wt)w t + A x (pa 2 Tw t ) + A y (-yw t ) - V y ■ (cw t ). (4.54) 



Remark 4.3 One obtains the existence and uniqueness of function-valued solutions of 
(|4,54p even if the initial measure wo is a degenerate one without density. 

Proof The proof follows the same steps as the one of Theorem 13.21 except that the 
mutation term will lead to a Laplacian term in / since the mutation kernel is centered and 
the mutation steps converge to in the appropriate scale. We first obtain the tightness of 
the sequence (Z K ) and the fact that each subsequence converges to a measure-valued flow 
w G C([0,T], Mf(X x satisfying for bounded C 2 -functions f,gi,g2, 



We can also apply wt to smooth time-dependent test functions h(t, x, y) defined on M + x 
X x . That will add a term of the form (d s h, w s ) in ([4.550 . 

Let us now sketch the uniqueness argument. Thanks to the Lipschitz continuity and 
ellipticity assumption (H3), the semigroup associated with the infinitesimal generator 



admits at each time t > a smooth density denoted by ip x ' y (t, ■, •) on X x That is, 
for any bounded continuous function G on X x , the function 




+ f{x)l(x) (A51 (2/1)52(3/2) A 52(2/2)) }w s (dx,dyi,y 2 )ds. 



(4.55) 



A := p a 2 T A x + 7 A y + c • V 



y 




satisfies 



-G = ^G; G(0,-,-) = G. 



28 



Thus (|4.55|) applied to the test function (s, x, y) i— > G(f — s, x, y) leads to the mild 
equation: for any continuous and bounded function G, 

(w t , G) = (w , G(t, •)) + [ (w s , (B-D-aU* w s )G(t - s, •)) ds 

Jo 

G(x',y') [ 4> x ' y (t,x',y')w (dx,dy) dx'dy' 

XxR\ JXxR\ 

+ J G{x',y') j j (B - D - aU *w s ){x,y)t(; x ' y (t- s,x',y')w s (dx,dy) dsdx'dy'. 

(4.56) 

It is simple to deduce from this representation the uniqueness of the measure-valued solu- 
tions of (|4,55p . Moreover, by Fubini's theorem and (H3) and since sup t<T {wt, 1) < +00, 
one observes that 

(w t ,G) = [ G(x',y')H t (x',y')dx'dy', 

JXxR\ 

with H e L 00 ^,! 1 ],!, 1 ^ x R\)). Thus for any t < T, the finite measure w t is absolutely 
continuous with respect to the Lebesgue's measure and the solution of (|4,55p is indeed a 
function for any positive time. □ 

If 77 = 1 the limiting process of Z K is no more deterministic but is a random superprocess 
with values in C([0,T],M F (X x M? + )). 

Theorem 4.4 Assume (H3) and r] = 1. Assume moreover that the initial conditions 
Zq £ Mp{X x M^_) satisfy sup K E((Zq , l) 3 ) < +00. // they converge in law as k tends 
to infinity to a finite deterministic measure £o> then the sequence of processes (^f)o<t<T 
converges in law in the Skorohod space B([0, T], Mp(X x M^_)), as k goes to infinity, to the 
continuous measure-valued semimartingale ( G C([0, T], Mp{X x satisfying for any 

bounded smooth functions f, 51,52' 

m( 9 := (CtJgm)- (Co, fgm) 

B{x,yi,y 2 ) - D(x,yi,y 2 ) - a(x,yi,y 2 ) U * Cs{x,yi,y 2 ) S j f{x)g 1 {yi)g 2 {y 2 ) 

+ p(x)a 2 (x)T(x,yi,y 2 ) A f(x)g 1 (y 1 )g 2 (y 2 ) 
+ f(x)V9i (51)52(2/2) (ri(x) - /3i(x)(yiAn + 52A12)) V\ 
+ f (x)g x (y\)^ g 2 (y 2 ){r 2 (x) - (3 2 (x)(y 1 X 21 + 52A22)) y 2 

+ f(x)j{x) (Ag 1 (51)52(52) +5i(5i) A 52(52)) yQsidx^yx^y^ ds (4.57) 



JXxR 2 + 

.2 



is a continuous square integrable {T^r^^-martingale with quadratic variation 

(M^) t = f [ 2T{x,y 1 ,y 2 )f 2 (x)g 2 l (yi)g 2 2 (y 2 )Udx,dy 1 ,dy 2 ) ds. (4.5* 

JO JXxR 2 , 
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Proof The convergence is obtained by a compactness- uniqueness argument. The uniform 
tightness of the laws and the identification of the limiting values can be adapted from [7] 
with some careful moment estimates and an additional drift term as in the proof of Theorem 
1531 

The uniqueness can be deduced from the one with B = D = a = 0by using the Dawson- 
Girsanov transform for measure- valued processes (cf. Theorem 2.3 in [5|), as soon as the 
ellipticity assumption for T is satisfied. Indeed, 

E \ I I (B(x,yi,y 2 ) - D(x, yi,y 2 ) - a(x,y 1 ,y 2 ) U* ( s (x,y 1: y 2 )\ ( s (dx, dyi, dy 2 )ds ) < +oo, 
which allows us to use this transform. 

In the case B = D = a = the proof of uniqueness can be adapted from the general results 
of Fitzsimmons, see [6] Corollary 2.23: the Laplace transform of the process is uniquely 
identified using the extension of the martingale problem (|4.57p to test functions depending 
smoothly on the time like (s,x,yi,y 2 ) i-> tpt-sf{x,y\,y2) for bounded functions / (see [6] 
Proposition 2.13). 

□ 
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